This application claims the benefit of Japanese Patent Application No. 08-357917, filed in Japan on Dec. 28, 1996, which is hereby incorporated by reference.
1. Field of the Invention
The present invention relates to a color reproduction correction device and method for an imaging apparatus, and more particularly to a technique wherein images of an object acquired by means of a color-image imaging device, such as a television camera or the like, can accurately be reproduced in accordance with the colors as seen by the operator of the imaging apparatus even if the light source used for illumination of the object varies.
2. Discussion of the Related Art
The human eye generally sees a white surface of an object as white regardless of the type of illumination involved. This property of the human eye is referred to as visual adaptation. In a color-image imaging apparatus, such as a television camera or the like, an adjustment is made in order to reproduce the visual adaptation of the human eye so that the output signals of the three primary colors used in imaging are the same when a white object is imaged regardless of the illumination. This technique is called white balance processing.
However, the three primary colors, red (R), green (G), and blue (B), used in imaging do not necessarily have the same characteristics as those exhibited by the human eye. Furthermore, the physical characteristics of the reference white color of the display devices displaying the results of the imaging differ from the illumination actually used in the imaging process. Because of these factors, colors other than achromatic colors may appear different from the colors of the actual object when the results of the imaging are viewed by means of a display device. In cases where the viewers have no knowledge of the colors at the imaging site, as in broadcasts, no problems exists as long as the colors match the colors in the memory of the viewers. However, when the colors themselves are critical, as in printed documents, or when the imaging is required to illicit a feeling of actual presence the difference between the object image and the displayed image colors requires improvement.
A linear matrix method is a conventional method employed to improve the color reproduction of color-image imaging devices, such as television cameras, used for broadcasting and similar applications. For example, in conventional television cameras, in which three types of color signals are produced, i.e., red (R), green (G), and blue (B), corrected red, green, and blue signals are obtained via a matrix circuit that mixes the signals with each other. These corrected signals are then subjected to signal processing, thus producing a video signal for television use.
However, procedures for determining the constants employed in matrices used for color reproduction correction have not always been firmly established. Historically the constants were selected by trial and error until visually pleasing results were obtained. Accordingly, linear matrix constants, which produce optimal results in cases wherein the light source used to illuminate the imaging object varies, have not been obtained. Therefore, linear matrix circuits are not used in most cases. Alternatively, color tones are adjusted by adjusting the hue and amount of color on the basis of visual determinations made by broadcast technicians using an imaging monitor.
In devices, such as home-use video cameras, video signals are obtained predominantly from a single imaging element utilizing three types of color filters that are applied to each pixel. However, since there are limits to the spectral characteristics of the filters, which can be mounted on the imaging element, the color reproduction is generally poor. Correction in those devices is generally accomplished by adjusting the hue and amount of color so that color tones, which are close to skin color (for which the eye has a high color discrimination sensitivity), will be favorable under the illumination most frequently used. Generally, variations in the illumination light source is handled only by white balance processing that adjusts the levels of the red, green, and blue, generally referred to as (RGB) signals so that the color white seen by the eyes is imaged as white.
Furthermore, a common problem associated with the respective devices described above is that there is no attempt to reproduce the appearance of colors according to the illumination used. The color reproduction is selected on the basis of whether the finished image is appealing to the viewer""s eye. Therefore, the atmosphere and/or feeling generated by the actual illumination is ignored. This problem occurs because no method has been established for determining correction matrices that takes into consideration the fact that since the reference white color of the television monitor of a display device does not necessarily coincide with the white color of the actual illuminating light that illuminates the imaging object. The reference white color displayed xe2x80x9cas isxe2x80x9d may differ from the color viewed directly by a live observer. Accordingly, it is necessary to investigate color reproduction by determining if imaging is performed under illumination from a white incandescent lamp or under outdoor sunlight, which heretofore has been ignored.
Accordingly, the present invention is directed to a color reproduction correction device that substantially obviates one or more of the problems due to limitations and disadvantages of the related art. An object of the present invention is to allow color reproduction correction, by means of a simple construction of a color reproduction correction device and method for an imaging apparatus, in which correction is accomplished by determining color reproduction correction matrices so that colors seen by the operator of the imaging apparatus can be accurately reproduced by a display device even if the light source used for illumination of the object varies.
Additional features and advantages of the invention will be set forth in the description which follows, and in part will be apparent from the description, or may be learned by practice of the invention. The objectives and other advantages of the invention will be realized and attained by the structure particularly pointed out in the written description and claims hereof as well as the appended drawings.
To achieve these and other advantages and in accordance with the purpose of the present invention, as embodied and broadly described, a color reproduction correction device for an imaging apparatus obtains color signals, with a plurality of spectral characteristics, by imaging an object and obtaining a color image. The color image is obtained by subjecting the color signals to signal processing, wherein the reproduction correction device includes a light source inferring device, which infers the type of illuminating light source being used, and a memory device, which stores, for each of a plurality of predetermined light sources, constants for correction matrix operations. The constants are determined so that the mean values of the absolute values of the distances between values obtained by converting the positions on a uniform color space of each of a plurality of colors of the object to be imaged illuminated by the respective light sources into the respective corresponding positions on the uniform color space in the case of white light from the reference light source of a display device that uses as reference white light a light source that may not coincide with the white light of the light source used for the illumination taking into consideration the adaptive effect of the eyes; the positions on the uniform color space of colors corresponding to the respective colors of the object to be imaged that are obtained when output signals produced by applying correction matrix operations to the color signals of the imaging device imaging the object are displayed by means of the display device, are minimized, thus insuring that the visual appearance of the plurality of colors of the object to be imaged under the illumination is approximately comparable to the visual appearance of the colors corresponding to the respective colors of the object to be imaged in an image obtained by displaying by means of the display device the output signals obtained by imaging the illuminated object by means of the imaging apparatus; and an operating device which extracts correction matrix constants, corresponding to the illuminating light source (inferred by the light source inferring device), from the memory device, and performs correction matrix operations on the color signals, at the time of imaging.
Thus, colors viewed by the human eye can be accurately reproduced by a display device if the visual stimulus color values of the object to be imaged are captured under the specific illumination and are converted into visual stimulus color values. Reproduction by the display device, the correction matrix constants, in the form of the visual stimulus color values, are employed to adjust the color values of the display device. These color adjustments approximate the color values associated with the specific illumination employed at the time of the object being imaged. Such a conversion of stimulus values is equivalent to the expression of color appearances under one illumination to those as seen under another illumination. Therefore, a model of the color adaptation of the human eye can be applied to such a conversion. Accordingly, reference stimulus values are obtained by measuring, for each of the plurality of light sources, the visual appearance of a plurality of colors of the object to be imaged as seen under the illuminating light source and converting these values into stimulus values that expresses the reference white color of the display device.
Correction matrix operations are then applied to the color signals of the imaging apparatus and the imaged stimulus values are determined from the resulting values. The stimulus values are then compared with the previously determined reference stimulus values and the constants of the correction matrices are determined so that the error between the respective stimulus values is minimized. The constants are then stored in the memory device. At the time of imaging, if the correction matrix constants of the illuminating light source, discriminated by a light source inferring device, are called up from the memory device and correction matrix operations are applied to the color signals, the desired color reproduction correction can be performed so that optimal color reproduction can be realized.
It is desirable that the light illuminating source inferring device infer the type of illuminating light source employed based on the white balance control signal that is utilized to perform white balance processing on the plurality of color signals.
Since the white balance control signal varies according to the spectral distribution of the illuminating light source, the type of the illuminating light source being used can easily be inferred and the construction of the device can also be simplified.
In another aspect of the present invention, color signals with a color reproduction correction device is equipped with a light source inferring device that infers the type of illuminating light source being used and a memory device that stores, for each of a plurality of predetermined light sources, constants for correction matrix operations.
The constants for correction matrix operations are determined as follows. The mean values of the absolute values of the distances between (a) and (b) are minimized. The (a) values are values obtained by converting the positions on a uniform color space, of each of a plurality of colors of the object to be imaged, illuminated by the respective light sources into the respective corresponding positions on the uniform color space in the case of white light from the reference light source of a display device, which uses as the reference white light a light source that may not coincide with the white light of the light source used for the illumination (taking into consideration the adaptive effect of the eyes). The (b) values are the positions on the uniform color space of colors corresponding to the respective colors of the object to be imaged that are obtained when output signals produced by applying correction matrix operations to the color signals of the imaging device are displayed by means of the display device. Thus the visual appearance of the plurality of colors of the imaging object is approximated comparable to the visual appearance of the colors of the imaging object under the respective illumination light.
Furthermore, a constant inferring device is employed. This device infers the correction matrix constants that approximate most closely the illuminating light source inferred by the light source inferring device by performing operations based on the constants stored in the memory device at the time of imaging. Finally, an operating device uses the constants output by the constant inferring device to perform correction matrix operations on the color signals at the time of imaging.
The constants of the correction matrices vary according to the type of illumination. Accordingly, it is impractical to prepare correction matrix constants for all possible types of illumination. Therefore, correction matrix constants are determined for several predetermined light sources and are stored in the memory device. The correction matrix constants, which most closely approximate the illuminating light source used, can be determined by the constant inferring device by interpolation from the values of the constants stored in the memory device. As a result, the construction of the device is simplified and accurate color reproduction correction can be efficiently performed.
It is desirable that the light source inferring device infer the type of illuminating light source used based on the white balance control signal that is used to perform white balance processing on the plurality of color signals. The white balance control signal has a spectral distribution that varies according to the type of illuminating light source used. Accordingly, the type of illuminating light source being used can be easily and accurately inferred from the white balance control signal, and the construction of the device can again be simplified.
Furthermore, in the correction matrix operations, each of the plurality of output signals following the correction operation can be expressed as the sum of one color signal of the plurality of color signals and values obtained by multiplying each of a plurality of difference signals between the plurality of color signals by a constant.
By applying correction matrix operations to the signals obtained by applying white balance processing to the plurality of imaged color signals, it is possible to ensure that white objects appear white. Also, a constraining condition, which prevents any variation in the white balance following correction, can be applied. Accordingly, the sum of the coefficients in the row direction of the correction matrices can be set equal to 1. Consequently, the outputs following correction can be obtained by summing the values obtained by applying correction matrix operations to the color difference signals such as (G-R), (G-B) and to the color signals prior to correction. In this way the number of variables required for color reproduction correction can be reduced, so that the construction of the device and the processing can further be simplified.
The device may also be constructed so that the color signals, with a plurality of spectral characteristics, are selected in such a manner that the variation in the constants applied to the difference signals between the plurality of color signals in the correction matrices of the respective color signals is minimized, even if the illumination utilized to illuminate the object varies. For example, if color filters with appropriate spectral characteristics are used, situations can occur in which illumination having different color temperatures will have little effect even if the amount of correction of the color difference signals is almost zero. In such cases the number of variables in the correction matrices can be reduced even further.
It is desirable to have the operating device perform correction matrix operations on a plurality of color signals obtained by performing white balance processing on the plurality of color signals. By performing correction matrix operations it is also possible to reduce the number of variables in the correction signals so that the construction of the device and the processing can be further simplified.
In a further aspect of the present invention a color reproduction correction method for an imaging apparatus that obtains color signals with a plurality of spectral characteristics by imaging an object to be imaged, and that obtains a color image by subjecting these signals to signal processing, including the steps of determining the constant values of correction matrix operation so that the value obtained by converting the position on a uniform color space of a given color of the object illuminated by a given light source into the corresponding position on the uniform color space when white light from the reference light source of a display device, which uses as reference white light, a light source that may not coincide with the white light of the light source used for the illumination (taking into consideration the adaptive effect of the eyes). Also, the position on the uniform color space of the color that is obtained when output signals produced by applying the correction matrix operations to the color signals of the imaging device imaging the object to be imaged are displayed by means of the display device are capable of approximating each other, thus insuring that the visual appearance of the color of the object under illumination is approximately comparable to the visual appearance of the color of the image obtained by displaying by means of the display device, wherein the output signals are obtained by imaging the illuminated object by the imaging apparatus.
In the first embodiment of the present invention such a method determines accurate constant values for correction matrix operations by simple processing.
In a still further aspect, a color reproduction correction method for an imaging apparatus that obtains color signals with a plurality of spectral characteristics by imaging an object to be imaged, and that obtains a color image by subjecting these signals to signal processing includes the steps of determining the constant value of correction matrix operation so that values obtained by converting the positions on a uniform color space of each of a plurality of colors of the object to be imaged illuminated by light source into the respective corresponding positions on the uniform color space in the case of white light from the reference light source of a display device which uses as reference white light a light source that may not coincide with the white light of the light source used for the illumination (taking into consideration the adaptive effect of the eyes). The positions on the uniform color space of colors corresponding to the respective colors of the object of imaging that are obtained when output signals produced by applying the correction matrix operation to the color signals of the imaging device imaging the object of imaging of the display device are minimized, thus ensuring that the visual appearance of the of colors of the object to be imaged under the illumination is approximately comparable to the visual appearance of the colors corresponding to the respective colors of the object of imaging in an image obtained by displaying by means of the display device the output signals obtained by imaging the illuminated object of imaging by means of the imaging apparatus.
The constant value of the correction matrix operation is determined so that the mean value of the absolute values of the distances are minimized for a plurality of colors of the object to be imaged. Therefore, color reproduction correction can be accurately performed for many colors, thus allowing higher-fidelity color reproduction.
The correction matrix operation may also be constructed so that each of the plurality of output signals following the correction operation is expressed as the sum of one color signal of the plurality of color signals and the values obtained by multiplying each of a plurality of difference signals between the plurality of color signals by a constant. Also, the sum of the coefficients, in the row directions of the correction matrices, can be set equal to 1 by performing the correction matrix operation on signals that have been subjected to white balance processing. Therefore, corrected outputs can be obtained as the sums of the values obtained by applying the correction matrix operation to two color difference signals and the color signals prior to correction. This technique reduces the number of variables required for color reproduction correction.
Furthermore, it is desirable that the color signals, with a plurality of spectral characteristics, be selected so that the variation in the constants applied to the difference signals between the plurality of color signals in the correction matrices of the respective color signals is minimized, even if the illumination of the object to be imaged varies.
By using color filters with the appropriate spectral characteristics, it is possible to reduce the variation in the constants thus applied to the difference signals between the plurality of color signals in the correction matrices of the respective color signals, thereby further reducing the number of variables in the correction matrices of the respective color signals.
Thus, in the present invention, by taking the color adaptation of the human eye into account, it is possible to optimize color reproduction by means of a simply constructed apparatus. Accordingly, colors that approximate the colors seen at the time of imaging can be reproduced in a display device.
Since color difference signals are used for correction, the number of correction coefficients required can be reduced and the apparatus can be simplified as well as reduced in the size. In addition, if appropriate spectral characteristics are applied to the imaging apparatus, some of the coefficients may be eliminated.
Additionally, by performing the correction processing, following white balance processing, it is possible to simplify the construction of the apparatus and to achieve more accurate correction of color during reproduction.
It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory and are intended to provide further explanation of the invention as claimed.